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Abstract 

Let G be the Heisenberg group of real lower triangular 3x3 matrices 
with unit diagonal. A locally free smooth action of G on a manifold M 4 
is given by linearly independent vector fields X\,X 2 ,X 3 such that A3 = 
[X U X 2 ] and [X U X 3 ] = [X 2 ,X 3 ] = 0. The C 1 topology for vector fields 
induces a topology in the space of actions of G on M 4 . An action is compact 
if all orbits are compact. Given a compact action 9, we investigate under 
which conditions its C 1 perturbations 6 are guaranteed to be compact. 
There is more than one interesting definition of stability, and we show that 
in the case of the Heisenberg group, unlike for actions of R n , the definitions 
do not turn out to be equivalent. 
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1 Introduction 



Let G be the Heisenberg group of real lower triangular 3x3 matrices with unit 
diagonal: G is a nonabelian nilpotent Lie group of dimension 3. Let Q be its Lie 
algebra of strictly lower triangular matrices. Let Mq be a smooth manifold of 
dimension 4: a locally free smooth action 9 : G x Mq — > Mo is given by linearly 
independent vector fields Xi,X 2 ,X 3 such that X 3 = [Xi,X 2 ] and [Ad,X 3 ] = 
[X2,X3] = 0. A Riemann metric on Mo induces a C l metric for vector fields on 
Mo and therefore a C 1 metric in the space of such actions. 

An action 9 is compact if all its orbits are compact. Is it true that all actions 
9 which are near 9 (in the C 1 metric) are also compact? This is true for some, but 
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not all, compact actions 9 and the best way to attack this problem is to consider 
first a neighborhood of an orbit and actions of a very special form. 

A given homomorphism : G — > G may be described as follows. Take the 
1-parameter subgroups a±, a 2 , ol 3 : R — > G, 

'\ 0\ /l 0\ /l N 

a 1 (t) = <j>\t 10, a 2 {t) = (t> 10, a 3 (t) = 1 



Let 




1/ \0 t 1/ W 1. 




n ;.((.)) = I a y 
^a 3j a 2j 

the numbers ajj are entries of a 3 x 3 matrix A with a 13 = a 23 = 0, a 33 = ana 2 2 — 
ai 2 a 2 i. Conversely, a matrix with these properties defines a homomorphism. The 
homomorphism <fi is an automorphism exactly when A is invertible. 

Let H C G be the group of matrices in G with integer coordinates: if is a 
discrete cocompact subgroup of G. Let M = (G/H) x (— e, e) where G/H = 
{gH,g G G} is a compact manifold of dimension 3. An action is homogeneous 
horizontal if it is of the form 

9: Gx M -> M 

(g,(9i H ,z)) ^ ((/>z(9)9iH,z) 

where <p z is a smooth family of automorphisms of G such that 0o is the identity. 
The orbits of 9 are of the form (G/H) x {z} and therefore compact. With the 
notation of the previous paragraph, 9 is described by a family of matrices A(z), 
z G (-e, e) satisfying A(0) = I, a 13 (z) = a 23 (z) = 0, a 33 (z) = a n (z)a 2 2(z) - 
ai 2 (z)a 2 i(z) . Let 

'a' u (z) a' 12 (zY 
a' 2l (z) a' 22 (z) / 



A\z) 



The orbit (G/H) x {0} of a homogeneous horizontal action 9 is (C 1 ) totally 
stable (or T-stable) if for any neighborhood U of (G/H) x {0} there is 5 > such 
that if X 1 ,X 2 , X 3 are smooth vector fields on U satisfying 

X 3 = [X U X 2 ], [X u Xz\ = [X 2 ,X 3 ] = 0, dc^Xj,Xj) < 5, (j = 1,2,3) 

then the orbit by Xi, X 2 , X 3 of any point of the form (*, 0) is compact. Otherwise, 
the orbit is T-unstable. 

The orbit (G/H) x {0} of a homogeneous horizontal action 9 is (C 1 ) locally 
unstable (or L-unstable) if for any 5 > there exist smooth vector fields Xi, X 2 , X 3 
in (G/H) x (— e, e) satisfying 

X 3 = [X!,X 2 ], [Xx, X 3 ] = [A 2 ,X 3 ] = 0, dci^-,^-) < 5, (j = 1,2,3) 
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such that Xj coincides with Xj outside (G/H) x (—5, 5) and such that the orbits 
by X 1 , X 2 , X 3 of any point of the form (*, 0) is noncompact. Otherwise, the orbit 
is L-stable. 

Our main theorems concern the stability of the orbit (G/H) x {0} of a ho- 
mogeneous horizontal action 6. 

Theorem 1 If A$(0) is nilpotent then the orbit is L-unstable. 
Theorem 2 If 

I -A 0\ 

A'(0) = 2A ,A ± 
\ OA/ 

then the orbit is L-stable and T-unstable. 

Theorem 3 7/det A s (0) ^ then the orbit is L-stable. 

Theorem 4 // A\0) has no real eigenvalues then the orbit is T-stable. 

In section 2 we justify our interest in homogeneous horizontal actions by 
proving that for any compact action, the neighborhood of an orbit admits a finite 
covering which is homogeneous horizontal. In section 3 we prove the instability 
claims, i.e., we present explicit perturbations of certain actions. The main result 
of section 4 is that a certain vector field is uniquely ergodic. In section 5 we study 
the underlying foliations. Finally, in section 6, we prove our main results. 

This paper follows closely the first author's thesis ([1]), except for a chapter 
which was developed separately ([2]). The simpler case of actions of W 1 was 
studied earlier by the second author ([10]). 

The authors acknowledge support from CNPq, CAPES and Faperj (Brazil). 
This work was done while the first author was in a leave of absence from Univer- 
sidade Federal do Para and had a visiting position in UFF. 



2 Homogeneous horizontal actions 

Let G be the Heisenberg group, H C G be the discrete subgroup of integer 
matrices, M be an arbitrary 4-manifold and M = G/H x (— e, e). 

Proposition 2.1 Let 9 : G x M — > M be a smooth locally free compact action. 
For any p G M there exists an open neighborhood V of p with the following 
properties: 

1. V is invariant under 9 ; 
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2. there exists a covering map of finite order ip : M — > V and a homogeneous 
horizontal action 9 on M such that 9 (g,ip(p)) = i[)(9(g,p)) for all g G G 
and p G M. 



Proof: Since orbits are compact and have codimension 1, the holonomy of each 
orbit of 6q contains at most two elements and admits a saturated open neigh- 
borhood (see [3] or [4]). Take V to be such a neighborhood. We assume the 
holonomy to be trivial; otherwise just take a double cover. 

Let 7 : (— e, e) — + M be a smooth curve transversal to 9 with 7(0) = Pq. For 
each z, the subgroup H z C G of isotropy of 7(2) is cocompact and discrete: let 
A z = H Z /[G,G] C G/[G,G] = Z 2 and a(z),b(z) G H z be continuously chosen 
elements projecting via the quotient by [G, G] to generators of A z . Let 4> z : G — > 
G be the automorphism with 

2 j^l 1 ojj=a(z), fa^jo 1 ojj=&(z) 

and define ^ and by ip((H,z)) = 7(2) and 0(<7i, {g^H^z)) = (<p~ 1 (gi)g2H, z) so 
that tp((gH,z)) = 6> o (0 2 (g), 7(2;)). We are left with proving that ^ is indeed a 
covering map of finite order and that 9 is well defined: these are rather straight- 
forward checks and are left to the reader. Notice that the order of ip as a covering 
is the index in H z of the subgroup generated by a(z), b(z). ■ 



3 Instability 

Points of G/H have coordinates (1/1,1/2,1/3), meaning 

1 

yi 1 

2/3 2/2 1 

so that (2/1,2/2,2/3) = (y[, 2/ 2 ,2/ 3 ) if and onl y if y[-yu V^-Vi and (s/3 — 2/a) — S/2 (s/i — 
2/i) are integers. Alternatively, G/iJ can be constructed from the cube [0, l] 3 
(with points (2/1,2/2,2/3)) by glueing faces as follows: (2/1,2/2,0) with (2/1,2/2,1); 
(2/1, 0, 2/3) with (2/1, 1, 2/3); (0, 2/2, 2/3) with (1, 2/2, 2/2+2/3)- The vector fields Y 1 = 
*2 = ^ + 2/i ^ and Y 3 = 4 are well defined in G/H and satisfy [YI, Y 2 ] = Y 3 , 
[Yi, Y 3 ] = [Y 2 , Y 3 ] = 0. Set Z = the vector fields Y 1? Y 2 , Y 3 , Z form a basis for 
the tangent space at any point of M = (G/H) x (— e, e). 
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3.1 Proof of theorem 1 

Our action 8 is given by a one parameter family of matrices A(z) with A(0) = I, 
A"(0) nilpotent, where 



A\z) 



a 'll(z) a 'l2( Z ) 

a' 21 (z) a' 22 {z) 



Set B = A'(0): notice that 613 = 623 = and 633 = 0.^(0)0-22(0) + oii(0)a 22 (0) — 
a' 12 (0)a 2 i(0) - ai 2 (0)a 21 (0) = a' n (0) + a' 22 (0) = trace A'(0) = 0. Write A(z) = 
I + zB: clearly, 033(2) = a u (z)a 22 (z) — a 12 (z)a 21 (z) for all z and this family of 
matrices therefore defines a family of automorphisms 2 : G — > G and an action 

(g,(9iH,z)) ^ (^ z (g)g!H,z) 

for which 

X 1 (y 1 ,y 2 ,y 3 ,z) = (1 + 2&n)Yi + 2621^2 + 2631*3 
-^2(2/1, 1/2, 1/3, 2) = zoi 2 Yi + (1 + zb 22 )Y 2 + 2632 13 

X 3 (yi,y2,y3,z) = Y 3 

Since # and 8 are C 1 close in a neighborhood of z = 0, it suffices to construct 
perturbations of 8. 

Let (p 2 , —pi) be a nonzero vector in the kernel of -A"(0), i.e., 

011P2 - 012P1 = 0, 
021P2 - 022P1 = 0. 

Let (gi, q 2 , q 4 ) G 1R 3 be a solution of the equation 

qiPi + Q2P2 + ^4(032^1 - 031P2) = 

with g 4 7^ (which is possible since pi and p 2 are not both zero) and define 
Cij = qiPj- Set 

Xx(yi, y 2 ,y 3 ,z) = Xi(y 1 ,y 2 , ys,z) + i/j(z) (cnYi + c 2 iF 2 + cu.Z) , 
X 2 (y\, y2, 1/3, z) = X 2 (y u y 2 , y 3 , z) + ^(z) (ci 2 Y"i + C22I2 + c A2 Z) , 
X 3 (yi, Z/2, 1/3, z) = X 3 (y u y 2 , y 3 , z), 

where ip is a bump function with ^(0) > 0. A straightforward computation 
verifies that 

[X U X 2 ] = X 3 , [X u X 3 ] = [X 2 , X 3 ] = 0, 

so that this defines an action. Since either C41 7^ or C42 7^ 0, orbits of this action 
intersecting z = are not compact. This is the required perturbation. 
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3.2 Proof of theorem 2 (instability part) 

As above, take an explicit action 9 with correct A'(0): 

Xi(yi,V2,V3,z) = e~ Xz Y l7 
X 2 { yi ,y 2 ,y 3 ,z) = e 2Xz Y 2 , 
X 3 ( yi ,y 2 ,y 3 ,z) = e Xz Y 3 . 

A straightforward computation verifies that, for 

Xi(y u y 2 , 1/3, z) = y 2 , y 3 , z), 

X 2 (yi, y 2 , y 3 , z) = X 2 (y 1: y 2 , 1/3, z) - ^e Az ^, 

X 3 (yi, 1/2, 1/3, z) = X 3 (y u 1/2, 1/3, z) - cYi, 

we have 

[X u X 2 ] = X 3 , [X u X 3 ] = [X 2 , X 3 ] = 

and this therefore defines a local action 9. For any nonzero c, the orbits of 9 
crossing z = are noncompact. This, for small c, is the required perturbation. 

4 Unique ergodicity of flows 
in a nilpotent manifold 

Let X = viYi + v 2 Y 2 + v 3 Y 3 with v 2 /v 1 £ Q: we prove that the vector field X is 
uniquely ergodic in G/H. 

In order to prove this result we consider the diffeomorphism : T 2 — > T 2 
defined by <p(z,w) = (az,zw) where a = e 2nc , c (jz Q (here we identify T 2 = 
S 1 x S 1 , S 1 C C): this is also uniquely ergodic. We employ the ideas of Van der 
Corput's theorem ([7]). 

4.1 A diffeomorphism of the torus 

Proposition 4.1 The diffeomorphism <fi : T 2 — > T 2 defined by<p(z,w) = (az,zw) 
where a = e 2nc , c Q, is uniquely ergodic. 

Notice that the usual unit measure \i (the Haar measure, a constant multiple 
of the Lebesgue measure) is invariant under 0. This proposition implies that all 
orbits of are dense in T 2 . 
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Proof: We must prove ([8] or Theorem 9.2 of [9]) that if / : T 2 — > C is continuous 
then the sequence 

9iV = ^(/ + /o^ + - + /o^- 1 ) 

converges uniformly to a constant (the integral of /). Furthermore, it suffices 
to consider / a Laurent polynomial in z and w. Indeed, let / be an arbitrary 
continuous function and let /„ be a sequence of Laurent polynomial tending 
uniformly to /. If a is the integral of / and a n is the integral of /„ we clearly 
have a n — > a. We may subtract a from / and a n from f n and assume without loss 
of generality that a = a n = 0. Given e > 0, let n be such that ||/ no — /||oo < e/2. 
We thus have 

^||(/no+/n o O0+--- + /„ o O0^ 1 )-(/ + /o0+... + /o0 JV ~ 1 )|| oo <£ 

for any N. Since f no is a Laurent polynomial, we assume that 

^||(/no + /n o O0+--- + /no°0 iV - 1 )L<| 

for sufficiently large N. This implies 

for large N, as desired. 

Since a Laurent polynomial is a sum of monomials, it is sufficient to prove 
our claim for f(z,w) = z n w m . It is easy to show by induction that 

/ o ^(z, w) = a jn+ y (j - 1)m z n+jm w m 
for all j. Our claim is trivial for n = m = 0. For m = and n^O, 

9n ( Zj w ) = ( z n + a n z n + a 2n z n + ■■■ + a {N ~ l)n z n ) 



N a n - I 

Since a and n are fixed and \z\ = \a\ = 1, (a Nn — l)/(a; n — l)z n is bounded and 
Qn tends uniformly to zero as iV goes to infinity. 

Finally, if m ^ we have 

g N (z, w) = ^(z n + a n z n+m + ■■■ + a jn+ ^ j -^ m z n+jm + 

|_ a {N-l)n+\(N-l)(N-2)m z n+{N-l)m^ w m ^ 
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Let w and z be the unit vectors in C given by 



z _ 1 ( z n a jn+\j{j-l)m z n+jm a (N -l)n+i(iV -1)(JV -2)m ^n+(W -l)m\ 



With the usual inner product, \qn(z,w)\ = |(z,w)| and we must prove that 
(z,w) tends to zero uniformly in z. Let Rn : C N — > be the linear trans- 
formation with R N (a , ai, . . . , ajv-2, ojv-i) — • • • , o>n-2, ajv-i, a>o)', we have 
R* N (a , ai, . . . , ajv-2, ojv-i) = (ajv_i,a ,ai, . . . ,ajv_ 2 ) and -R^w = w so that 
(R^z,w) = (z, w). A straightforward computation yields 



«z,z) = -( 



1 /a^ N ^ km — 1 



a fcm - 1 



|(JV-fc)(iV-fe-l)m j , _-i(AT-l)(JV-2)m\_W-fc)(-n)_(iV-fc)(-m) 



_|_ ( Q; -2V J,- ' i 'A J, -«'-V" t _|_ . . . _|_ Q;^2 , - 1V_ - L A JV -^"M Q , 



for fixed k, the expression between parenthesis on the right hand side is uniformly 
bounded in z and therefore, for any fixed k, (R%z, z) tends to zero uniformly in 
z. 

Let e > and M > 4/e 2 . From what we just proved, there exists N > M 
such that 

\(R N z,z)\,\(R%z,z)\,...\(R%-\z)\<j } 
for all N > N . Then 

\z + R N z + ... + R™- X z\ = ^((z + • • • + R^z), (z + • • • + 



< ^M+^(M 2 -M) < V2M. 



Thus 



| (z, w) I = -1 1 (z + R N z + ■ ■ ■ + R%~\ w) 

< ■^|z + J R iV z+..- + J R^- 1 z||w| 

< — V2M = \ — < t 
~ M V M 



for all N > N and for all z. 
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4.2 Vector fields on G/H 



Theorem 5 Let X be the vector field on G/H, 

X = v-lYx + v 2 Y 2 + v 3 Y 3 

where vi, v 2 , v 3 are constant real numbers. lfv 2 /v\ is irrational then X is uniquely 
ergodic. 

Notice that this implies that every orbit of X is dense in G/H. 

Proof: Assume without loss of generality that v\ = 1. Let : tx G/ H — > G / H 
be the flux corresponding to X: 

<f>(t, (yi, 1/2, y3)) = (yi + t,y 2 + v 2 t, y 3 + v 3 t + v 2 y x t + )^ 2 t 2 ). 

Let m be the Lebesgue or Haar measure on G/H: it is clear that X preserves m. 
Given a Borel set A and x G G/H, let 

t(x,A) = lim h({t e [0,T\\<Kt,x) e A}) 

where A is the Lebesgue measure on R. In order to prove that X is uniquely 
ergodic it suffices to prove that t(x, A) = m(A) for every x G G/H and every A 
in a basis of the a-algebra of Borel sets. Our basis consists of sets of the form 

A = A e>yuD = {<f)(s, (y 1} j/2, y 3 )), s G (0, e), (y 2 , y 3 ) G D} 

where e G (0, 1) is a real number and D C T 2 is a Borel set (possibly a disc): 
m(A e ^ yi jj) = €/jl(D) where \x is the unit Lebesgue measure on T 2 . 

Let to be the smallest nonnegative real number for which (f>(to,x) is of the 
form (yi,*,*): set (fi(t ,x) = (yi,x), x G T 2 . Clearly, 

t(x, A et y uD ) = lim ^ card{s e Z,0 < s < T, 4>{(x) e D} 

' ' T— >oo ± 

where 0i : T 2 — > T 2 is defined by 0; more precisely, 

0(1, (yi, 1/2, l/s)) = (yi, 0i (1/2, 1/3)) = (3/1,3/2 + «2, 1/3 - 1/2 + ^3 + (l/i - l/2)u 2 )- 

The diffeomorphism of the torus 0i is conjugate to a diffeomorphism of the form 
discussed in 4.1 and therefore uniquely ergodic. Thus r(x, A emt o) = ^^{D) = 
m{A) and we are done. ■ 
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5 Foliations 



5.1 Excellent foliations and 

near-identity nilpotent pseudogroups 

We define in this section excellent foliations of M — G/H x (— e, e), which have 
noncompact orbits but are near compact foliations. We then define the horizontal 
subalgebra at each point; we use results from [2]. The reader should also see [6] 
for the more natural problem of studying nilpotent groups of diffeomorphisms of 
one-manifolds. 

Let be the foliation of M whose leaves are of the form G/H x {*}. Thus, 
JF is the foliation defined by a horizontal homogeneous action. We will study 
foliations T near JF . 

We call T acceptable if its leaves are always transversal to vertical lines, i.e., 
lines of the form {*} x (— e, e). Clearly, foliations near T§ are acceptable. Alter- 
natively, we say that a foliation is acceptable if the restriction of the projection 
ttm '■ M — > G/H to a leaf is a local diffeomorphism. 

Given a function 7 : [a, b] — > G with 7(a) = g and a point (goH, z ) G M, 
we call the lift of 7 to be a path of the form 7 : [a, a + 5) — > M such that 
7(a) = (g H,z ), 7'(t) is tangent to T for all t and ttm 7 — 7, where 
tth '■ G — > Gr/if is the canonical projection. Clearly, lifts exist for sufficiently 
small 5 and there exists a unique maximal lift; we say that 7 lifts if the domain of 
the maximal lift 7 is [a, b]. The foliation T is said to be k-good if 7 lifs provided 
\z \ < e/2 and the length of the image of 7 is smaller than k; abusing notation, 
we say that T is good if it is /c-good for some implicitly given value of k. 

For a good foliation T and go G G, define the functions f^ : (—e/2, e/2) — > 
(— e, e) as follows. First, for i = 1, 2, 3, s = ±1, construct paths 7* o i : [0, 1] — > G 
defined by 



Now, for z G (—e/2, e/2), lift each 7* o ^ starting from (g H,z) to obtain points 
(goH, fg Q ^z)). Clearly, each fg gi is smooth and strictly increasing. Notice that 
is the inverse of fg o i ; from now on, we write f 90i i instead of fg o i . Moreover, 
if g H = g x H then f gu3 = f g0t3 and, for i = 1, 2, 



for integers 711,712: this means that the choice of go is not strongly relevant and 
from now on g will be omitted in the notation. Finally, 




ft 




(/go, 3) 



[/ 2 , /1] = o fr 1 o f 2 o /1 = / 3 , [/ s , /1] = [/ 3 , / 2 ] = id. 
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The functions /i, f 2 , /3 form a near-identity nilpotent pseudogroup of functions 
as denned in [2] if they are sufficiently near the identity; in this case, we call T 
excellent. A leaf C is called abelian if fz(z) = z for (goH, z) e £; the foliation J 7 
is called abelian if all its leaves are abelian, or, equivalently, if f'3 is the identity. 

Let a : G — > R be a group homomorphism: the 1-form da can be lifted 
to G/H and therefore to G/H x R, where it is closed but probably not exact. 
The 1-form dz, where z is the R-coordinate, is exact in G/H x R and therefore 
da + dz is a closed 1-form in G/H x R, defining a foliation T a - We say that 
the homomorphism a is rational if it takes elements of H to rational numbers 
and a is irrational otherwise. The theorem below is a slightly modified version 
of theorem 2 from [2]. 

Theorem 6 Given a manifold of the form G/H there exists €g/h such that, if 
T\ is a smooth foliation of G/H x (—1, 1) with d c i(.Fo, < ^g/h, then one of 
the three conditions hold: 

1. T\ has at least one compact leaf and is an abelian foliation; for any maximal 
connected open set U of G/H x (—1, 1) containing no compact leaf there 
exists a homomorphism a : G — > R, an open set J C G/H x R and a 
homeomorphism $ : J — > [/ taking T a to T\ . 

2. T\ has no compact leaf and is an abelian foliation; there exists an irrational 
homomorphism a : G — > R, an open set J" C G/H x R and a homeomor- 
phism $ : J" — > G/if x (—1, 1) taking T a to T\. 

3. T\ has no compact leaf and has an abelian leaf £ ; there exists a rational ho- 
momorphism a : G — > R, an open set J" C G/H x R anc? a homeomorphism 
$ : J7" — > G/iJ x (—1, 1) taking one leaf Cq of T a to a leaf Cq. 

The construction of the homeomorphism $ is such that the G/H coordinate 
is preserved, i.e., $(gH,*) = (gH,*). 

5.2 The horizontal subalgebra 

For a group homomorphism a : G — > R, let a' : Q — > R be the induced Lie algebra 
homomorphism. We call the kernel £ C Q of a' the horizontal subalgebra. This 
can also be contructed as follows: let a be the translation number of f 2 relative 
to fi, the horizontal subalgebra £ C Q is defined as the set of matrices 
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In order to present another interpretation for £ we first construct a local action 
F : G x M -> M. 

For p = (gH, z ) £ M and h E G, let 5 : [0, 1] — > G be a continuous 
path with 5(0) = /, 5(1) = h. Construct a path 7 : [0, 1] — > M whose image 
is contained in a leaf of JF (or, equivalently, such that "f'(t) is tangent to JF 
for all i) such that 7(0) = p , 7(4) = (5(t)gH,*); define F(/i,p ) = 7(1)- It 
is easy to prove independence on the choice of 5 and that F is indeed a local 
action, i.e., that F(/i 1 /i 2 ,Po) = F(/i 1; F(/i 2 ,Po))- Let \& be the derivative of F, 
i.e., ^/ Po (f) = F(exp(t> ), po). It follows easily from the theorem above that if po 
is sufficiently far from the boundary of M then fy po (v) is defined for all v £ £\, 
where £\ is the closed neighborhood of radius 1 of £; by compactness, for such 
p , the function ty po : £ 1 — > M is uniformly continuous. 

From now on we shall consider two cases, depending on whether a is a rational 
or irrational homomorphism, or, equivalently, depending on whether a, the trans- 
lation number of / 2 relative to f\, is rational or irrational. The irrational case 
corresponds either to case 2 in theorem 6 or to case 1 with a irrational. In the ir- 
rational case, the closure of *f> po (£) is M l po = $(*, zq) where $ _1 (po) — {gH, z ), 
a topological manifold homeomorphic to G/H. The vector field —aYi + Y 2 is 
tangent to M ljP0 and, from theorem 5, uniquely ergodic in M l po . The topological 
manifold M^ Po thus receives a unit measure \i. 

Consider the (^1,3/2) torus T 2 = IR 2 /Z 2 , where R 2 has coordinates y\ and 
?/2- In the rational case, the projection of ty po (£) C IR 2 onto T 2 is a rational 
curve and the projection onto G/H is a 2-dimensional torus. If po belongs to an 
abelian leaf (for instance, in case 1), then ty po (£) is a 2-dimensional torus M 2jP0 
contained in $(*,^o) where $ _1 (po) = (gH,zo). If po belongs to a nonabelian 
leaf, however, there may exist infinitely many points (g H, z) in ty po (£) above each 
point gH of the torus. For each gH, let z + and z~ be the supremum and infimum, 
respectively, of the set of values of z for which (gH,z) belongs to ty po (£). The 
tori M£ and M 2 ~ po are formed by points of the form (gH,z + ) and (gH,z~), 
respectively, and are contained in abelian noncompact orbits. We shall usually 
take po as an abelian leaf to avoid these complications but in either case the 
vector fields Y 3 and —aYi + Y 2 are tangent to M 2 m or M^ po and define unique 
unit invariant measures /x or /i 1 * 1 in M 2 po or . 

If po = (gH,zo) and pi = (gH,zi) belong to the same leaf then projection 
yields a measure preserving homeomorphism L POtPl : M* iPo — > M* iPl . 
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6 Stability 



6.1 The function TAb 

Let Ab : G — > R 2 and Ab' : Q — > M? be the abelianization of the group G and its 
Lie algebra Q, respectively, i.e., the projections defined by 



( yi ,y 2 ), Ab' U =(ui,U2). 





Let 6* the homogeneous horizontal action 

9: Gx M -> M 

(g,(9iH,z)) i-> ((f> z (g)giH,z) 

where M = G/H x (— e, e) and 2 is a smooth family of automorphisms of G 
such that 0o is the identity. The orbits of 9 form the horizontal foliation JF . Let 
be a smooth local action near 9; its orbits form a foliation JF of M. If 9 is 
a perturbation of 9, we may assume T to be an excellent foliation and use the 
results of the previous section. For a vector v G Q, write f Po (v) for the vector w 
in C/ taken by 9 to a vector of the form (v, *): 

T(*) = 0(exp(H,Po), T'(0) = («,*). 
Given p 0) define TAb, Po : £ — > by 




Ab'(f p (v))dn(p) 



where M* )Po = Mi jPo , M %Po = M 2iPo or M* iPo = M^ po depending on whether the 
translation number is irrational or rational. 

In order to provide a geometric interpretation for the function TAb, we intro- 
duce an auxiliary function £ po : S\ — > G: 

£ po is uniformly continuous in E\. 

Lemma 6.1 For a noncompact perturbation 9 of 9, 

TAh, P0 {v) = lim ^Ab(^ Po (tv)). 

t— >oo i 



13 



Proof: We first consider the irrational case. Take 

/0 0\ 

v = (vi,v 2 ,v 3 ) = Ivi G S; 

\v 3 v 2 0/ 

we have Ab(£ Po (tv)) = J* Ab'(f p ^(v ))ds where p(s) = ^ Po (sv). If (i>i,i>2) 7^ it 
follows from theorem 5 and from Birkhoff 's theorem that 

lim ^-Ab(£ Po (tv)) = I Ab'(f p (v))dii(p) = r AhtPo (v), 
t Jm 1 , po 

as required. If v = (0, 0, 1), we have 

lim ^Ab(£ po (tv)) = [ Ab'(f p {v))dp, 

where S(p ) C Mi is the circle F(exp(t(0, 0, l)),p ). We have to show that if 
Pi G Mi then 



Ab'(f p (v))dp= ! Ab'(f p (v))dp, 

JS(pi) 



S(po) JS( P1 ) 

i.e., that 

lim -Ab(Z P0 (tv)) = lim -Ab(£ P0 (tv + w)) 

where v G £ and w G S\. Since we have already seen that both limits exist, it 
sufficies to prove that Ab(£ Po (tv )) — Ab(£ Po (tt> + w)) is bounded, and this follows 
from uniform continuity of £ Po . 

In the rational case, M^ Po is a torus. If the orbit of v is dense, the result 
follows directly from Birkhoff's theorem, as before. Otherwise, we again have 
that the limit equals an integral over a circle and independence from the choice 
of circle follows from uniform continuity of £ Po . ■ 



6.2 Dependence of TAb, Po on po 
Set 

/o 

£3= 
\1 

Lemma 6.2 The value of r Ah p (E 3 ) is constant on each abelian orbit of 9. If 9 
has a compact orbit then TAb, P (E 3 ) = for any point p. 
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Proof: We first prove that Ab(£ p (E 3 )) is constant on abelian orbits. Indeed, 
let p = (goH, z ) and p 1 = {qqH, z±) belong to the same abelian orbit of 9 with 
^(fiSPo) = Pi- The closed loops 7 R)i 3 and 7 Pl; 3 are the images under 9 of paths 
70,71 e G with -1 7i(*)0 = 7o(t). Clearly, £^(£3) = 7,(1) at p<, i = 0, 1 and 
therefore Ab{^ Po {E 3 )) = Ab(£ Pl (E 3 )) . From the proof of lemma 6.1, it follows 
that this constant value of Ab(£ p (E 3 )) is T AhtP0 (E 3 ). 

At a compact orbit, the path 7 P0j3 is the commutator of 7 P0) i and 7 P0i2 and 
therefore £ P0 (E 3 ) is also a commutator and therefore Ab(^ Po (E 3 )) = r Ah:Po (E 3 ) = 
0. On the other hand, if po belongs to a noncompact abelian orbit but a com- 
pact orbit exists then there exist pi,p 2 , . . . is the same orbit as po tending to 
Poo, a point in a compact orbit. We have Ab(C, Pi (E 3 )) = Ab(^ P0 (E 3 )) for all %. 
On the other hand, the closed loops 7 Pi! 3 tend uniformly to 7 Poo ,3 and therefore 
lin W ,Ab(^(£ 3 )) = Ab{i Poo {E 3 )) = and we have T Ah , P0 (E 3 )°°= Ab(£ P0 (E 3 )) = 
0. ■ 

The example presented in subsection 3.2 shows that TAb iPo (E 3 ) is not always 
zero, not even if the foliation is abelian. In that example, it is easy to verify that 



C(gH,z)(E 3 ) = 




and therefore T Ah ^ gHyZ) (E 3 ) = (c, 0) ^ 0. 

Lemma 6.3 If po = (goH,z ) and pi = (g H,zi) belong to the same abelian 
orbit of 9 then r A b, Pl {v) - r AhyPo (v) is a multiple ofr AhyP (E 3 ). 



Proof: Set p\ = F(h,p ) and F h be the local diffeomorphism F h {p) = F(h,p). 
For u E S, write u h = h^uh, the conjugate of u by h. We clearly have 
hexp(tu h ) = exp(tu)h and therefore F h (^ po (tu h )) = ^ Pl (tu). From the definition 
of f, we have F h (9(£ Po (tu h ),p )) = 0(£ Pl (tu),pi) and therefore \£ Po (tu h ) -£ Pl {tu)\ 
is bounded when t goes to infinity, yielding T A \ hPo {u h ) = T A ^ tPl (u). Since u h — u 
is a multiple of E 3 we are done. ■ 

The following two results are now immediate. 

Corollary 6.4 If 9 has a compact orbit then r A ^, p is constant in each connected 
component of the complement of compact orbits. 

Corollary 6.5 The image of the map r AhjP has dimension 1 and is constant 
in each connected component of a vertical line of the form (goH, *) minus the 
intersection with compact orbits. 
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6.3 Proof of theorems 3 and 4 



We begin with theorem 3. Assume by contradiction that there exists a non- 
compact action 9 near 9 coinciding with 9 outside a neighborhood of an orbit. 
Take po = (goH, z ) and pi = (goH, zi), z\ > zo, in the same non-compact orbit. 
From corollary 6.4, TAb, P0 = TA.b,pi- Let v G £ with |Ab'(i>)| = 1. By definition of 
T"Ab, P , we have 

r A b, P » = / Ah'(f p (v))dfi(p) 

JM*. V . 



and therefore 

/ (Ab'(f Wp) (*;)) - Ab\f p {v)))d^p) = 0. 
On the other hand, for each p e M* )P0 , 

Ah'(rr(M) ~ Ab '(^(^)) = Ab ' ( r 4-A 9 H, z) dz ) (v). 



zo dz 



Notice that for the original action 9, 

Ab% Ht , ) {v) = B{z)(Ab'(v)), B{z) 
and therefore 



au{z) a 12 (z) 
a 2 i(z) a 22 (z) 



Kh Tz f{9H ' z){v) W -( 5 (^)^(^)5(^))(Ab'(t;)). 

Since A$(z) is assumed to be invertible, — (B(z)A^(z)B(z))(Ab'(v )) 7^ 0; by com- 
pactness, for 9 sufficiently near 9, Ab' -^?( gH:Z )(v ) belongs to a convex open set 
not containing the origin and therefore the integral above belongs to the open 
cone spanned by this open set, contradicting the fact that the integral equals 
zero. This completes the proof of theorem 3. 

For the proof of theorem 4, assume without loss of generality that A\u)/\u > 
for all nonzero vectors u. We know that if T Ah)Po (v) = w then TAb, P1 (v) = cw for 
some positive number c. We have w A TAb, P1 (v) = and therefore 

w A Ab' ^ jT -^f(gH,z)dz^j (v) = 0. 

For 9 sufficiently near 9, we may assume that Ab'(f p (f)) always belongs to 
a narrow angle around w; since B(z) is near the identity we have w A 
(— (B(z)A i (z)B(z))(Ab'(v ))) > 0, again yielding a contradiction. 
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